Artificial general intelligence (AGI) is an established field of research. Yet Melanie Mitchell and others have questioned if the term still has meaning. AGI has been subject to so much hype and speculation it has become something of a Rorschach test. Mitchell points out that the debate will only be settled through long term, scientific investigation. To that end here is a short, accessible and provocative overview of AGI. I compare definitions of intelligence, settling on intelligence in terms of adaptation and AGI as an artificial scientist. Taking my queue from Sutton's Bitter Lesson I describe two foundational tools used to build adaptive systems: search and approximation. I compare pros, cons, hybrids and architectures like o3, AlphaGo, AERA, NARS and Hyperon. I then discuss overall meta-approaches to making systems behave more intelligently. I divide them into scale-maxing, simp-maxing, w-maxing based on the Bitter Lesson, Ockham's and Bennett's Razors. These maximise resources, simplicity of form, and the weakness of constraints on functionality. I discuss examples including AIXI, the free energy principle and The Embiggening of language models. I conclude that though scale-maxed approximation dominates, AGI will be a fusion of tools and meta-approaches. The Embiggening was enabled by improvements in hardware. Now the bottlenecks are sample and energy efficiency.

This study examines the alignment of \emph{Conference on Computer Vision and Pattern Recognition} (CVPR) research with the principles of the "bitter lesson" proposed by Rich Sutton. We analyze two decades of CVPR abstracts and titles using large language models (LLMs) to assess the field's embracement of these principles. Our methodology leverages state-of-the-art natural language processing techniques to systematically evaluate the evolution of research approaches in computer vision. The results reveal significant trends in the adoption of general-purpose learning algorithms and the utilization of increased computational resources. We discuss the implications of these findings for the future direction of computer vision research and its potential impact on broader artificial intelligence development. This work contributes to the ongoing dialogue about the most effective strategies for advancing machine learning and computer vision, offering insights that may guide future research priorities and methodologies in the field.

In a popular blog post titled "The Bitter Lesson," Richard Sutton argues that AI's progress has resulted from cheaper computation, not human design decisions based on problem-specific information. Sutton diminishes researchers that build knowledge into solutions based on their understanding of a problem to improve performance. This temptation, Sutton explains, is good for short-term performance gains, and such vanity is satisfying to the researcher. However, such human ingenuity comes at the expense of AI's divine destiny by inhibiting the development of a solution that doesn't want our help understanding a problem. AI's goal is to recreate the problem-solver ex nihilo, not to solve problems directly.[1]

It states that general learning methods that can scale with computation are ultimately the most effective. The two methods that can seemingly scale endlessly are search and learning, and they have bore their fruit. Sutton lists out their successes in chess, go, speech recognition, computer vision, etc, etc. This is in contrast to the human-knowledge approach, where our knowledge of a specific domain is built into the algorithms that are trying to "solve" or "work-out", so to speak, that domain. In speech recognition, this was with the hand crafting of phonemes, words, etc; in games like chess/go this was through crafting for features of the game; and the list goes on and on.

The biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin. The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation. Most AI research has been conducted as if the computation available to the agent were constant (in which case leveraging human knowledge would be one of the only ways to improve performance) but, over a slightly longer time than a typical research project, massively more computation inevitably becomes available. Seeking an improvement that makes a difference in the shorter term, researchers seek to leverage their human knowledge of the domain, but the only thing that matters, in the long run, is the leveraging of computation. These two need not run counter to each other, but in practice, they tend to.